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> 

>T!TLE:  Visual  Communications:  The  Next  Generation  -  Compression, 

>  Access,  and  Delivery  of  Visual  Information 

> 


> 

>  SUMMARY 

> 

>This  ARO-YIP  grant  resulted  in  the  creation  of  novel  paradigms  for  the 
representation,  compression,  and  delivery  over  noisy  channels  and 
>heterogeneous  networks  of  images  and  video.  Several  publications  resulted 
>from  this  week,  and  this  work  formed  the  core  of  two  M.S.  and  one  Ph.D. 
>theses.  A  brief  summary  of  the  projects  follows  in  the  following 
>sections  of  this  final  report. 

> 

> 

>1)  Hybrid  analog/digital  framework  for  Source-Channel  Coding  of  images 
> 

>  The  aim  of  this  project  was  to  consider  the  problem  of  image 

>  transmission  over  wireless  channels  using  joint  source-channel 

>  coding.  Inspired  by  both  information-theoretic  principles  and 

>  realistic  wavelet-based  image  models,  we  have  advocated  a  novel 

>  practical  wavelet-based  framework  for  integrating  analog  and 

>  digital  modes  of  communication  for 

>  transmitting  images  over  noisy,  time-varying  channels.  Our 

>  goal  was  to  maximize  the  end-to-end  delivered  image  quality 

>  subject  to  constraints  on  power  and  bandwidth.  We  considered  a 

>  hierarchy  of  increasingly  more  sophisticated  statistical 

>  wavelet-based  image  models.  Our  proposed  joint  source-channel 

>  coding  algorithm,  founded  on  a  hybrid  analog-digital  framework,  is 

>  validated  both  theoretically  by  its  attractive  proximity  to  the 

>  information-theoretic  bounds  on  the  underlying  statistical  image 

>  model  used,  as  well  as  empirically  by  its  excellent  performance  on 

>  real  natural  images.  Our  results  indicate  possibly  significant 

>  performance  gains,  of  the  order  of  3  dB  in  PSNR,  compared  to 

>  conventional  state-of-the-art  "all  digital"  approaches  to  joint 

>  source-channel  coding.  Figure~\ref{system}  gives  a  block  diagram  of 

>  the  conventional  and  hybrid  systems.  Details  can  be  found 

>  in  [1]- 

> 

> 

> 

>2)  Region-based  video  coding  using  mathematical  morphology 
> 

>  The  motivation  in  this  project  was  to  use  ideas  from  mathematical 

>  morphology  for  region-based  video  coding.  Motion  estimation  and 

>  compensation  has  always  been  a  critical  problem 

>  for  video  coding.  Traditional  video  coders  use  block-based  motion 

>  compensation,  which  is  simple  and  regular  but  causes  blocky 

>  artifacts.  New  generation  video  coding  standards  such  as  MPEG-4 

>  calls  for  object-based  approaches,  which  require  understanding  of 

>  the  image 

>  semantics.  In  this  project,  we  segmented  the  video  frames  into 

>  distinct  regions,  which  intuitively  correspond  to  moving  physical 

>  objects.  Then  we  estimated  and  coded  the  motion  for 

>  each  region.  Regions  could  have  arbitrary  shape,  and  were  found  by 

>  "growing"  from  a  "seed"  using  morphological  operations. 

> 

>  A  striking  feature  of  this  coder  is  that  the  segmentation  is 

>  based  on  decoded  information,  therefore  region  shapes  or  contours 

>  do  not  need  to  be  coded.  Regions  can  be  merged,  pruned,  propagated, 

>  and  modified  from  frame  to  frame.  The 
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>  coder  performs  among  the  best  of  current  state-of-the-art 

>  region-based  video  coders,  and  outperforms  MPEG-1  at  around  1M  bps 

>  (bits  per  second).  Details  of  this  work  can  be  found  in  [2], 

> 

> 

> 

>3)  Multiresolutional  motion  estimation  and  video  coding  based  on  wavelets 
> 

>  Our  objective  in  this  project  was  to  improve  upon  existing 

>  motion  estimation  schemes  and  tranform  coding  methods  to  come  up 

>  with  a  video  coder  better  than  the  state-of-the-art.  We  attempted 

>  to  exploit  the  motion  correspondences  in  a  multiresolutional  fashion. 

>  Specifically,  we  first  code  a  coarse  resolution  version  of  the  video 

>  frame,  estimate  a  motion  field  from  this  coarse  resolution,  then 

>  apply  it  to  predict  the  next  finer  resolution.  Finally  we  coded  the 

>  predicted  difference  of  the  finer  resolution,  then  repeated  the  whole 

>  process  for  the  next  finer  resolution,  and  so  on.  Wavelet  transform 

>  provided  the  right  tool  for  this  framework,  because  of  its  inherent 

>  multiresolutional  nature. 

> 

>  In  our  work,  it  is  found  that  motion  relationship  indeed  bear 

>  considerable  coherence  across  different  resolution  levels,  but 

>  directly  estimating  them  from  wavelet  coefficients  is  not  feasible. 

>  A  major  problem  is  the  aliasing  noise  resulted  from  downsampling 

>  operations  in  the  transform.  This  problem  is  attacked  by  upsampling  and 

>  filtering  the  coarser  signals  using  a  specially  designed  interpolation 

>  filter. 

> 

>  The  coder  can  operate  in  two  modes:  A  purely  backward  mode  in  which 

>  no  motion  information  is  coded.  A  backward/forward  hybrid  mode  in 

>  which  the  encoder  judiciously 

>  chooses  to  send  motion  information  for  certain  areas  or  at  certain 

>  resolutions  for  each  frame,  and  the  decisions  are  optimized  using 

>  zerotree  coding  and  dynamic  programming.  The 

>  complete  coder  is  efficient,  scalable,  and  robust  over  large  range 

>  of  bitrates.  It  is  compared  against  MPEG-1 ,  at  high  bitrates 

>  (0.5-1 .5  Mb/s),  and  with  H.263  at  low  bitrates 

>  (24-128  kb/s).  It  achieves  typically  a  coding  gain  of  1  dB  over 

>  both  standards.  See  [3]  for  details. 

> 

> 

> 
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